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Amendments to the Claims: 

This listing of claims replaces all prior versions and listings of claims in the application: 
Listing of Claims : 
1.-57. (Cancelled) 

58. (Currently Amended) A system comprising: 
one or more computers; and 

a computer-readable medium coupled to the one or more computers having instructions 
stored thereon which, when executed by the one or more computers, cause the one or more 
computers to perform operations comprising: 

providing a multi-modal user interface on a mobile device, the multi-modal 
interface for performing inventory management tasks in a warehouse environment, and the 
multi-modal interface enabling output and user input of information using both first and second 
modalities that collectively include a voice-based modality and a touch-based modality ; 

receiving a first user input identifying a job type mode, the first user input being 
input using either the first modality or the second modality as chosen by a user of the mobile 
device , and the job type mode being selected from among a stocking mode in which the user 
enters one or more items into a bin in the warehouse environment , a picking mode in which the 
user removes the one or more items from the bin in the warehouse environment , and a counting 
mode in which the user counts the one or more items in the bin in the warehouse environment , 

transmitting information identifying the user-selected job type mode to an 
inventory management system a server , in a format consistent with the modality in which the 
first user input was received, 

responsive to transmitting the information, receiving a response to the information 
identifying the user-selected job type mode from the inventory management system server , in 
formats consistent with both the first modality and the second modality, the response identifying 
a location at which the user will perform a function related to the user-selected job type mode on 
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the one or more items, a description of the one or more items, and a best route from a present 
location of the user to the bin an item , 

providing the response to the user via the using both audio and display 
components of the multi-modal user interface, in the formats consistent with both the first 
modality and the second modality, 

receiving a second user input using either the first modality or the second 
modality as chosen by the user , the second user input indicating a completion of that the user has 
completed the selected function, and including count information or location information 
associated with the one or more items as observed by the user and, when the voice-based 
modality is used, a phonetically distinct checkword associated with the bin to indicate 
completion of the selected function item, 

updating inventory data for the one or more items item based on the count 
information or the location information, the updated inventory data being accessible using the 
formats consistent with both the first modality and the second modality, and 

transmitting information indicating the completion of that the user has completed 
the selected function related to the user-selected job type mode to the inventory management 
system , in a format consistent with the modality in which the second user input was received. 



59. (Currently Amended) The system of claim 58, wherein the operations further comprise: 
outputting the updated inventory data in a format consistent with either the first modality 

or the second modality, such that the inventory data is maintained during performance of the 
inventory management tasks. 

60. (Previously Presented) The system of claim 58, wherein the job type mode comprises job 
data. 



61 . (Currently Amended) The system of claim 58, wherein the multi-modal interface further 
enables output and user input information using first modality and the second modality 
collectively include two or more of a voice modality, a Radio Frequency Identification Device 
(RFID) modality, or a bar code modality^-a 
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62. (Currently Amended) The system of claim 58, wherein the operations further comprise 
receiving a pick list identifying the item one or more items and the function. 

63. (Previously Presented) The system of claim 62, wherein the operations further comprise: 
displaying the pick list using a visual user interface, and 

outputting, using voice synthesis, information identifying less than all items on the pick 

list. 

64. (Previously Presented) The system of claim 58, wherein the formats consistent with both 
the first modality and the second modality comprise HTML and VXML, respectively. 

65. -67. (Cancelled) 

68. (Currently Amended) The system of claim 58 further comprising: 

determining, based on updating the inventory data, that a-bin the bin associated with the 
item is expected to be empty; 

prompting the user to confirm whether the bin is actually empty, using both the first 
modality and the second modality. 

69. (Previously Presented) The system of claim 58, wherein the first user input is received 
using a different modality than the second user input. 

70. (Currently Amended) The system of claim 69, wherein: 

the first user input is received using a voice the voice-based modality, and 
the second user input is received using a tactile the touch-based modality. 



7 1 . (Currently Amended) A computer-readable medium encoded with a computer program 
comprising instructions that, when executed, operate to cause a computer to perform operations 
comprising: 
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providing a multi-modal user interface on a mobile device, the multi-modal interface for 
performing inventory management tasks in a warehouse environment, and the multi-modal 
interface enabling output and user input of information using both first and second modalities 
that collectively include a voice-based modality and a touch-based modality ; 

receiving a first user input identifying a job type mode, the first user input being input 
using either the first modality or the second modality as chosen by a user of the mobile device , 
and the job type mode being selected from among a stocking mode in which the user enters one 
or more items into a bin in the warehouse environment , a picking mode in which the user 
removes the one or more items from the bin in the warehouse environment , and a counting mode 
in which the user counts the one or more items in the bin in the warehouse environment ; 

transmitting information identifying the user-selected job type mode to an inventory 
management system a server , in a format consistent with the modality in which the first user 
input was received; 

responsive to transmitting the information, receiving a response to the information 
identifying the user-selected job type mode from the inventory management system s e rv e r , in 
formats consistent with both the first modality and the second modality, the response identifying 
a location at which the user will perform a function related to the user-selected job type mode on 
the one or more items, a description of the one or more items, and a best route from a present 
location of the user to the bin an item ; 

providing the response to the user via the using both audio and display components of the 
multi-modal user interface, in the formats consistent with both the first modality and the second 
modality; 

receiving a second user input using either the first modality or the second modality as 
chosen by the user , the second user input indicating a completion of that the user has completed 
the selected function, and including count information or location information associated with 
the one or more items as observed by the user and, when the voice-based modality is used, a 
phonetically distinct checkword associated with the bin to indicate completion of the selected 
function item; 
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updating inventory data for the one or more items item based on the count information or 
the location information, the updated inventory data being accessible using the formats 
consistent with both the first modality and the second modality; and 

transmitting information indicating the completion of that the user has completed the 
selected function related to the user-selected job type mode to the inventory management system , 
in a format consistent with the modality in which the second user input was received. 

72. (Currently Amended) The computer-readable medium of claim 71 , wherein the 
operations further comprise: 

outputting the updated inventory data in a format consistent with either the first modality 
or the second modality, such that the inventory data is maintained during performance of the 
inventory management tasks. 

73 . (Previously Presented) The computer-readable medium of claim 7 1 , wherein the job type 
mode comprises job data. 

74. (Currently Amended) The computer-readable medium of claim 71 , wherein the multi- 
modal interface further enables output and user input information using first modality and the 
second modality collectively include two or more of a voice modality, a Radio Frequency 
Identification Device (RFID) modality, or a bar code modalit y, a touch modality, and a visual 

75. (Currently Amended) The computer-readable medium of claim 71 , wherein the 
operations further comprise receiving a pick list identifying the item one or more items and the 
function. 



76. (Previously Presented) The computer-readable medium of claim 75, wherein the 
operations further comprise: 

displaying the pick list using a visual user interface, and 
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outputting, using voice synthesis, information identifying less than all items on the pick 

list. 

77. (Previously Presented) The computer-readable medium of claim 71, wherein the formats 
consistent with both the first modality and the second modality comprise HTML and VXML, 
respectively. 

78. -80. (Cancelled) 

8 1 . (Currently Amended) The computer-readable medium of claim 7 1 wherein the 
operations further comprise: 

determining, based on updating the inventory data, that a bin the bin associated with the 
item is expected to be empty; 

prompting the user to confirm whether the bin is actually empty, using both the first 
modality and the second modality. 

82. (Previously Presented) The computer-readable medium of claim 71 , wherein the first 
user input is received using a different modality than the second user input. 

83. (Currently Amended) The computer-readable medium of claim 82, wherein: 
the first user input is received using a voice the voice-based modality, and 
the second user input is received using a tactile the touch-based modality. 

84. (Currently Amended) A computer-implemented method comprising: 

providing a multi-modal user interface on a mobile device, the multi-modal interface for 
performing inventory management tasks in a warehouse environment, and the multi-modal 
interface enabling output and user input of information using both first and second modalities 
that collectively include a voice-based modality and a touch-based modality ; 

receiving a first user input identifying a job type mode, the first user input being input 
using either the first modality or the second modality as chosen by a user of the mobile device . 
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and the job type mode being selected from among a stocking mode in which the user enters one 
or more items into a bin in the warehouse environment , a picking mode in which the user 
removes the one or more items from the bin in the warehouse environment , and a counting mode 
in which the user counts the one or more items in the bin in the warehouse environment ; 

transmitting information identifying the user-selected job type mode to an inventory 
management system a server , in a format consistent with the modality in which the first user 
input was received; 

responsive to transmitting the information, receiving a response to the information 
identifying the user-selected job type mode from the inventory management system server , in 
formats consistent with both the first modality and the second modality, the response identifying 
a location at which the user will perform a function related to the user-selected job type mode on 
the one or more items, a description of the one or more items, and a best route from a present 
location of the user to the bin an item ; 

providing the response to the user via the using both audio and display components of the 
multi-modal user interface, in the formats consistent with both the first modality and the second 
modality; 

receiving a second user input using either the first modality or the second modality as 
chosen by the user , the second user input indicating a completion of that the user has completed 
the selected function, and including count information or location information associated with 
the one or more items as observed by the user and, when the voice-based modality is used, a 
phonetically distinct checkword associated with the bin to indicate completion of the selected 
function item; 

updating inventory data for the one or more items item based on the count information or 
the location information, the updated inventory data being accessible using the formats 
consistent with both the first modality and the second modality; and 

transmitting information indicating the completion of that the user has completed the 
selected function related to the user-selected job type mode to the inventory management system , 
in a format consistent with the modality in which the second user input was received. 
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85. (Currently Amended) The method of claim 84, further comprising: 

outputting the updated inventory data in a format consistent with either the first modality 
or the second modality, such that the inventory data is maintained during performance of the 
inventory management tasks. 

86. (Previously Presented) The method of claim 84, wherein the job type mode comprises 
job data. 

87. (Currently Amended) The method of claim 84, wherein the multi-modal interface further 
enables output and user input information using first modality and the second modality 
collectively include two or more of a voice modality, a Radio Frequency Identification Device 
(RFID) modality, or a bar code modalit y, a touch modality, and a visual modality . 

88. (Currently Amended) The method of claim 84, further comprising receiving a pick list 
identifying the item one or more items and the function. 

89. (Previously Presented) The method of claim 88, further comprising: 
displaying the pick list using a visual user interface, and 

outputting, using voice synthesis, information identifying less than all items on the pick 

list. 

90. (Previously Presented) The method of claim 84, wherein the formats consistent with both 
the first modality and the second modality comprise HTML and VXML, respectively. 

91. -93. (Cancelled) 

94. (Currently Amended) The method of claim 84 further comprising: 

determining, based on updating the inventory data, that a bin the bin associated with the 
item is expected to be empty; 
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prompting the user to confirm whether the bin is actually empty, using both the first 
modality and the second modality. 

95. (Previously Presented) The method of claim 84, wherein the first user input is received 
using a different modality than the second user input. 

96. (Currently Amended) The method of claim 95, wherein: 

the first user input is received using a voice the voice-based modality, and 
the second user input is received using a tactile the touch-based modality. 



